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Abstract. — In [10] it is proved that any morphism from a subgroup of finite index of SL(n,Z) to the group of 
analytic diffeomorphisms of § has a finite image as soon as n > 5. The case n = 4 is also claimed to follow along 
the same arguments; in fact this is not straightforward and this case indeed needs a modification of the argument. In 
this paper we recall the strategy for /; > 5 and then focus on the case n = 4. 
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1. Introduction 

After the works of Margulis ([14, 19]) on the linear representations of lattices of simple, real Lie groups 
with M-rank larger than 1 , some authors, like Zimmer, suggest to study the actions of lattices on compact 
manifolds ([20, 21, 22, 23]). One of the main conjectures of this program is the following: let us consider 
a connected, simple, real Lie group G and let T be a lattice of G of R-rank larger than 1. If there exists 
a morphism of infinite image from Y to the group of diffeomorphisms of a compact manifold M, then the 
]R-rank of G is bounded by the dimension of M. There are a lot of contributions in that direction ([3, 4, 5, 7, 
8, 9, 10, 11, 16, 17]). In this article we will focus on the embeddings of subgroups of finite index of SL(n,Z) 
into the group Diff Q) (S 2 ) of real analytic diffeomorphisms of S 2 {see [10]). 

The article is organized as follows. First of all we will recall the strategy of [10] : the study of the nilpo- 
tent subgroups of Diff ffl (S 2 ) implies that such subgroups are metabelian. But subgroups of finite index of 
SL(«,Z), for n > 5, contain nilpotent subgroups of length n — 1 of finite index which are not metabelian; as 
a consequence Ghys gets the following statement. 

Theorem A ([10]). — Let The a subgroup of finite index ofSL(n, Z). As soon asn>5 there is no embedding 
ofT into Diff D (S 2 ). 

To study nilpotent subgroups of Diff C0 (S 2 ) one has to study nilpotent subgroups of DiffJ ) (S 1 ) {see §2) and 
then nilpotent subgroups of the group of formal diffeomorphisms of C {see §3). The last section is devoted 
to establish the following result. 
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Theorem B. — Let T be a subgroup of finite index ofSh(n, Z). As soon as n > 4 there is no embedding ofT 
into Diff°(S 2 ). 

The proof relies on the characterization, up to isomorphism, of nilpotent subalgebras of length 3 of the 
algebra of formal vector fields of C 2 which vanish at the origin. 

Acknowledgements. — The author would like to thank Dominique Cerveau and Etienne Ghys for interest- 
ing discussions and advices. 



2. Nilpotent subgroups of the group of analytic diffeomorphisms of S 1 

Let G be a group; let us set 

G (0) =G & G (i) = [G,G (/ - 1) ] V/> 1. 

The group G is nilpotent if there exists an integer n such that G^ n > = {id}; the length of nilpotence of G is the 
smallest integer k such that GW = {id}. 
Set 

G( ) = G & G(j) = [G(,-_i),G( f _i)] V/ > 1. 

The group G is solvable if G/„) = {id} for a certain n; the length of solvability of G is the smallest integer k 
such that Gm = {id}. 

We say that the group G (resp. algebra g) is metabelian if [G,G] (resp. [g,g]) is abelian. 
Proposition 2.1 ([10]). — Any nilpotent subgroup o/Diff^(S ) is abelian. 

Proof. — Let G be a nilpotent subgroup of DiffJ ) (S 1 ). Assume that G is not abelian; it thus contains a 
Heisenberg group 

(f,g,h\[f,g]=h,[f,h] = [g,h}=^)- 

The application "rotation number" 

Diff^fS 1 ) -+R/Z, «f4 lim ¥ " (X) ~ X 

is not a morphism but its restriction to a solvable subgroup is ([1]). Thus the rotation number of h is zero and 
the set Fix(h) of fixed points of h is non-empty and finite. Considering some iterates of / and g instead of / 
and g one can assume that / and g fix any point of Fix(/i). The set of fixed points of a non trivial element of 
(/, g) is finite and invariant by h so the action of (f, g) is free on each component of S 1 \Fix(/i). But the 
action of a free group on E is abelian: contradiction. □ 



1. The stabilizer of every point is trivial, i.e. the action of a non trivial element of {/, g) has no fixed point. 
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3. Nilpotent subgroups of the group of formal diffeomorphisms of C 2 

Let us denote Diff(C 2 ,0) the group of formal diffeomorphisms of C 2 , i.e. the formal completion of the 
group of germs of holomorphic diffeomorphisms at 0. For any i let Diff, be the quotient of Diff(C 2 ,0) by 
the normal subgroups of formal diffeomorphisms tangent to the identity with multiplicity i; it can be viewed 
as the set of jets of diffeomorphisms at order i with the law of composition with truncation at order i. Note 
that Diff, is a complex linear algebraic group. One can see Diff(C 2 ,0) as the projective limit of the Diff,'s: 
Diff(C 2 ,0) = limDiff,. Let us denote by %(C 2 ,0) the algebra of formal vector fields in C 2 vanishing at 0. 

One can define the set %; of the /-th jets of vector fields; one has lim%, = %(C 2 ,0). 

Let 0(C 2 ) be the ring of formal series in two variables and let K(C 2 ) be its fraction field; O,- is the set of 
elements of 0(C 2 ) truncated at order i. 

The family (exp ( -: — > Diff;). is filtered, i.e. compatible with the truncation. We then define the expo- 
nential application as follows: exp = limexp^ : %(C 2 ,0) — > Diff(C 2 ,0). 

As in the classical case, if X belongs to %(C 2 ,0), then exp(X) can be seen as the "flow at time 1" of X. 
Indeed an element X of can be seen as a derivation of O,-; so it can be written Sj+Nj where S; and N,- are 
two semi-simple, resp. nilpotent derivations which commute. Taking the limit, one gets X = S + N where S 
is a semi-simple vector field and N a nilpotent one and [S,N] = id (see [15]). A semi-simple vector field is a 
formal vector field conjugate to a diagonal linear vector field which is complete. A vector field is nilpotent if 
and only if its linear part is; let us remark that the usual flow % of a nilpotent vector field is polynomial in t 

q> t (*)=£p 7 (?y, Pi^(C[t}) 2 
i 

so cpi (jc) is well defined. As a consequence exp(?X) = e\p(tS)exp(tN) is well defined for t = 1. Note that 
the Jordan decomposition is purely formal: if X is holomorphic, S and N are not necessary holomorphic. 

Proposition 3.1 ([10]). — Any nilpotent subalgebra o/x(C 2 ,0) is metabelian. 

Proof. — Let [ be a nilpotent subalgebra of %(C 2 ,0) and let Z(l) be its center. Since %(C 2 ,0) ®K(C 2 ) is a 
vector space of dimension 2 over ^(C 2 ) one has the following alternative: 

• the dimension of the subspace generated by Z(l) in %(C 2 ,0) ® A"(C 2 ) is 1; 

• the dimension of the subspace generated by Z(t) in %(C 2 ,0) ®K(C 2 ) is 2. 

Let us study these different cases. 

Under the first assumption there exists an element X of Z([) having the following property: any vector field 
of Z([) can be written uX with u in K(C 2 ). Let us consider the subalgebra g of [ given by 

= {X G [| 3u E K(C 2 ),X = uX). 

Since X belongs to Z((), the algebra g is abelian; it is also an ideal of [. Let us assume that I is not abelian: 
let Y be an element of I whose projection on [/g is non trivial and central. Any vector field of [ can be written 
as uX + vY with u, v in K(C 2 ). As X belongs to Z([) and Y is central modulo g one has 

X{u) =X(v) =Y(v) = 0. 

The vector fields ^ and ^ being some linear combinations of X and Y with coefficients in ,K(C 2 ,0), the 
partial derivatives of v are zero so v is a constant. Therefore [I, I] C g; but g is abelian so I is metabelian. 
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In the second case Z(l) has two elements X and Y which are linearly independent on K(C 2 ). Any vector 
field of I can be written as uX + vY with u and v in K(C 2 ). Since X and Y belong to Z(l) one has 

X{u) = X{v) = Y(u) = Y{v) = 0. 

As a consequence u and v are constant, i.e. I C {uX + vY \ u, v £ C}; in particular [ is abelian. □ 

Proposition 3.2 ([10]). — Any nilpotent subgroup o/Diff(C 2 ,0) is metabelian. 

Proof. — Let G be a nilpotent subgroup of Diff(C 2 ,0) of length k. Let us denote G, the projection of G 
on Diff,. The Zariski closure G, of G, in Diff, is an algebraic nilpotent subgroup of length k. It is sufficient to 
prove that G, is metabelian. 

Since G, is a complex algebraic subgroup it is the direct product of the subgroup G !jM of its unipotent 
elements and the subgroup G^ s of its semi-simple elements {see for example [2]). 

An element of Diff; is unipotent if and only if its linear part, which is in GL(2,C), is; so G,-, s projects 
injectively onto a nilpotent subgroup of GL(2,C). Therefore G,-, iS is abelian. 

Let us now consider G,-,„; this group is the exponential of a nilpotent Lie algebra [,■ of %, of length k. Taking 
the limit one thus obtains the existence of a nilpotent subalgebra [ of %(C 2 ,0) of length k such that exp([) 
projects onto G ! H for any i. According to Proposition 3. 1 the subalgebra [ and thus G,,„ are metabelian. □ 

4. Nilpotent subgroups of the group of analytic diffeomorphisms of S 2 

Proposition 4.1 ([10]). — Any nilpotent subgroup o/'Diff Q) (S 2 ) has a finite orbit. 

Proof. — Let G be a nilpotent subgroup of Diff B (S 2 ); up to finite index one can assume that the elements 
of G preserve the orientation. Let (j) be a non trivial element of G which commutes with G. Let Fix((j)) be the 
set of fixed points of (]); it is a non empty analytic subspace of S 2 invariant by G. If p is an isolated fixed point 
of (j), then the orbit of p under the action of G is finite. So it is sufficient to study the case where Fix((j)) only 
contains curves; there are thus two possibilities: 

• Fix((|)) is a singular analytic curve whose set of singular points is a finite orbit for G; 

• Fix((j)) is a smooth analytic curve, not necessary connected. One of the connected component of S 2 \ Fix((|)) 
is a disk denoted B. Any subgroup Y of finite index of G which contains (]) fixes B. Let us consider an 
element y of T and a fixed point m of y which is in B. By construction (]) has no fixed point in B so 
according to the Brouwer Theorem {§ k {m))k has a limit point on the boundary 3D of B. Therefore y has 
at least one fixed point on 3D. The group T thus acts on the circle 3B and any of its elements has a fixed 
point on B. Then Y has a fixed point on 3B (Proposition 2.1). 

□ 

Theorem 4.2 ([10]). — Any nilpotent subgroup o/Diff^S 2 ) is metabelian. 

Proof. — Let G be a nilpotent subgroup of Diff°(S 2 ) and let T be a subgroup of finite index of G having a 
fixed point m (such a subgroup exists according to Proposition 4.1). One can embed T into Diff(]R ,0), and so 
into Diff(C 2 ,0), by considering the jets of infinite order of elements of Tin m. According to Proposition 3.2 
the group T is metabelian. 

One can assume that G is a finitely generated group. 

Let us first assume that G has no element of finite order. Then G is a cocompact lattice of the nilpotent, 
simply connected Lie group G <8> M (see [18]). The group G is metabelian if and only if G <8> M is; but T is 
metabelian so G <g> K also. 



REPRESENTATIONS OF SOME LATTICES INTO THE GROUP OF ANALYTIC DIFFEOMORPHISMS OF THE SPHERE S 2 



5 



Finally let us consider the case where G has at least one element of finite order. The set of such elements 
is a normal subgroup of G which thus intersects non trivially the center Z(G) of G. Let us consider a non 
trivial element (j) of Z(G) which has finite order. Let us recall that a finite group of diffeomorphisms of the 
sphere is conjugate to a group of isometries. Denote by G + the subgroup of elements of G which preserve the 
orientation. It is thus sufficient to prove that G + is metabelian; indeed if (]) does not preserve the orientation, (|) 
has order 2 and G = Z/2Z x G + . So let us assume that (]) preserves the orientation; (j) is conjugate to a direct 
isometry of S 2 and has exactly two fixed points on the sphere. The group G has thus an invariant set of two 
elements. By considering germs in the neighborhood of these two points, one gets that G can be embedded 
into 2 • Diff(]R 2 ,0) ^ and thus into 2 • Diff(C 2 ,0): 

1 — >Diff(C 2 ,0) — >2-Diff(C 2 ,0) — >Z/2Z — >0. 

Let us remark that 2 • Diff (C 2 , 0) is the projective limit of the algebraic groups 2 • Diff;. The end of the proof is 
thus the same as the proof of Proposition 3.2 except that the subgroup of the semi-simple elements of 2 • Diff, 
embeds now in 2 • GL(2, C); it is metabelian because it contains an abelian subgroup of index 2. □ 

Let r be a subgroup of finite index of SL(n,Z) for n > 5. Since T contains nilpotent subgroups of finite 
index of length n — 1 (for example the group of upper triangular unipotent matrices) which are not metabelian 
one gets the following statement. 

Corollary 4.3 ([10]). — Let Y be a subgroup of finite index o/SL(n,Z); as soon as n > 5 there is no embed- 
ding ofT into Diff B (S 2 ). 

5. Nilpotent subgroups of length 3 of the group of analytic diffeomorphisms of S 2 

Let us precise Proposition 3.1 for nilpotent subalgebras of length 3 of %(C 2 ,0). Let [ be such an algebra. 
The center Z([) of [ generates a subspace of dimension at most 1 of %(C 2 ,0) <g> ^(C 2 ), for else [ would be 
abelian (Proposition 3. 1) and this is impossible under our assumptions. So let us assume that the dimension of 
the subspace generated by Z(l) in %(C 2 ,0) <^K(C 2 ) is 1. There exists an element X in Z([) with the following 
property: any element of Z([) can be written uX with u in K(C 2 ). Let g denote the abelian ideal of [ defined 
by 

Q = {Xe l\3u £K (C 2 ),X = uX). 

By hypothesis [ is not abelian. Let Y be in [; assume that its projection onto 1/q is a non trivial element 
of Z([/g). Any vector field of I can be written 

uX + vY, u,v£K(C 2 ). 

Since X, resp. Y belongs to Z(l), resp. Z(l/g) and since the length of [ is 3, one has 

X(u) =Y 3 (u) =X(v) =y(v) =0. (5.1) 

If X and Y are non singular, one can choose formal coordinates x and y such that X = and Y = j-. The 
previous conditions can be thus translated as follows: v is a constant and u is a polynomial in y of degree 2. 
We will see that we have a similar property without assumption on X and Y. 



2. Let G be a group and let q be a positive integer; q ■ G denotes the semi-direct product of G q by TLjqL under the action of the 
cyclic permutation of the factors. 



6 



JULIE DESERTI 



Lemma 5.1. — Let X and Y be two vector fields of %(C 2 ,0) that commute and are not colinear. One 
can assume that (X,Y) = y^,^~J where x and y are two independent variables in a Liouvillian extension 
ofK(C 2 ,0). 

Proof. — Since X and Y are non colinear, there exist two 1-forms a, [3 with coefficients in K(C 2 ) such that 

oc(X) = l, a(y)=0, PP0 = 0, P(X) = 1. 

The vector fields X and Y commute if and only if a and [3 are closed (this statement of linear algebra is 
true for convergent meromorphic vector fields and is also true in the completion). The 1-form a is closed so 
according to [6] one has 



!=1 



?2 



where \j/2 and the (j), denote some formal series and the X; some complex numbers. One has a similar 
expression for p\ So there exists a Liouvillian extension K of K(C 2 ) having two elements J and y with a = dx 
and f3 = dy. One thus has 



X(x} = \, 



X(y)=0, 



Y(x)=0, 



Y(y) = l. 



□ 



From (5.1) one gets: v is a constant and u is a polynomial in y of degree 2; so one proves the following 
statement. 

Proposition 5.2. — Let I be a nilpotent subalgebra of %(C 2 ,0) of length 3. Then [ is isomorphic to a subal- 
gebra of 

n =Ip(y)l- + a^ aeC,PeC[y],degP = 2). 
{ ox ay J 

Remark 5.3. — We use a real version of this statement whose proof is an adaptation of the previous one: a 
nilpotent subalgebra [ of length 3 of %(]R 2 , 0) is isomorphic to a subalgebra of 

a a 



n= |p(^_ + a — aeR,P£M[y],degP = 2}. 



dy 

Theorem 5.4. — Let T be a subgroup of finite index o/'SL(?2,Z); as soon as n > 4 there is no embedding 
ofT into Diff Q) (S 2 ). 

Proof. — Let U(4,Z) (resp. U(4,]R)) be the subgroup of unipotent upper triangular matrices of SL(4,Z) 
(resp. SL(4,R)); it is a nilpotent subgroup of length 3. Assume that there exists an embedding from a 
subgroup r of finite index of SL(4,Z) into Diff n (S 2 ). Up to finite index T contains U(4,Z). Let us 
set H = p(U(4,Z)). Up to finite index H has a fixed point (Proposition 4.1). One can thus see H as a 
subgroup of Diff(M 2 ,0) C Diff(M 2 ,0) up to finite index. 

Let us denote j l the morphism from Diff(IR 2 ,0) to Diff,. Up to conjugation j 1 (p(U(4,Z))) is a subgroup 

of 

q £ keM*,<eK}. 
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Up to index 2 one can thus assume that j op takes values in the connected, simply connected group T defined 
by 

\ [ X 

Let us set 

Diff ! -(T) = {/GDiff,|j 1 (/)GT}; 
the group Diff, (T) is a connected, simply connected, nilpotent and algebraic group. The morphism 

p ; -: U(4,Z) ^ Diflu- 
can be extended to a unique continuous morphism p ; : U(4,R) — > Diff,(T) (see [13, 12]) so to an algebraic 
morphism ( 3 ). Let us note that p,(U(4,Z)) is an algebraic subgroup of Diff ; (T) which contains p,(U(4,Z)); 
in particular H, = p,(U(4,Z)) C p,-(U(4,R)). By construction the family (H,), is filtered; since the extension 
is unique, the family (p,) ; is also filtered. Therefore K = limH, is well defined. Since p is injective, H is a 

nilpotent subgroup of length 3; as H C K and as any H, is nilpotent of length at most 3 the group K is nilpotent 
of length at most 3. For i sufficiently large p ! (U(4, R)) is nilpotent of length 3; this group is connected so its 
Lie algebra is also nilpotent of length 3. Therefore the image of 

Dp — limDp, : u(4,R) -> x(R 2 ,0) 

is isomorphic to n (Proposition 5.2). So there exists a surjective map \|/ from u(4,R) onto n. The kernel of \|/ 
is an ideal of u(4,R) of dimension 2; hence ker\|/ = (Si4,aSi3 +^824) where the 8y denote the Kronecker 
matrices. One concludes by remarking that dimZ(u(4,R)/ker\|/) = 2 whereas dimZ(n) = 1. □ 

Corollary 5.5. — The image of a morphism from a subgroup o/SL(n,Z) of finite index to Diff t0 (S 2 ) is finite 
as soon as n > 4. 



X, t G 
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